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(54) Composite display apparatus 

(57) A composite display apparatus has a display 
device for displaying image information, a display opti- 
cal system for guiding light from the display device to 
the eye of an observer, and an image pickup optical sys- 
tem for focusing light from the outside on an image 
pickup device. An optical axis of the image pickup opti- 
cal system is substantially aligned with an axis as an 
extension of the eye-ball optical axis in an optical path of 
the display optical system. The apparatus further has an 
optical path separating element for separating an optical 
path of the image pickup optical system from the optical 
path of the display optical system. Each of the display 
optical system and the image pickup optical system has 
a plurality of decentered reflective surfaces having cur- 
vature. The light from the cfisplay device is alternately 
folded in directions along the eye-ball optical axis by the 
plurality of reflective surfaces of the cfisplay optical sys- 
tem to be guided to the eye. The light from the outside is 
alternately folded in directions different from an obser- 
vation direction by the plurality of reflective surfaces 
including the optical path separating element The 
image pickup optical system has a stop and a rear opti- 
cal system placed behind the stop, and the image 
pickup optical system has a surface having a negative 
power and a surface having a positive power in order 
toward the stop in the optical path before the stop. The 
rear optical system is a refracting optical system or a 
reflecting optical system having a positive power. 
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Description 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001] The present invention relates to a composite 
display apparatus incorporating an image pickup optical 
system for taking in light from image information of the 
outside to form an image thereof on an image pickup 
device and a display optical system for permitting an 
observer to observe the image information displayed on 
a display device such as a liquid crystal panel or the like, 
which is constructed in compact size as a whole, and, 
more particularly, to such apparatus suitably applicable 
to devices called head-mounted displays and goggle- 
type displays. 

Related Background Art 

[0002] The devices suggested heretofore include a 
device constructed in such ' structure that a compact 
CCD camera is mounted on a head-mounted display 
(HMD) and image information obtained from the com- 
pact CCD camera is displayed in the HMD as it is or a 
device constructed in such structure that the image 
information obtained from the compact CCD camera is 
processed and converted into some image information 
and the image information thus converted is displayed 
in the HMD. In addition, there are also a variety of sug- 
gestions about a seethrough HMD (the HMD having 
structure for permitting the light from the outside to be 
observed directly at the angular magnification 1) as a 
type of the HMD. 

[0003] The HMD of this type will suffer parallax if the 
eye-ball optical axis of the display optical system for 
observing the image information displayed on the dis- 
play device of the HMD is offset against the optical axis 
of the CCD camera for imaging the image information of 
the outside on the CCD surface. The donees having 
both the image pickup optical system and the display 
optical system without the parallax are suggested, for 
example, in Japanese Laid-open Patent Applications 
No. 4-22358 and No. 5-303053. 
[0004] In the display optical system suggested in Jap- 
anese Laid-open Patent Application No 4-22358, light 
emitted from a compact liquid crystal panel is guided via 
a flat half mirror and thereafter to the eye by an eye- 
piece. On the other hand, the image pickup optical sys- 
tem is arranged so that the light from the outside is 
guided via aflat half mirror and thereafter is focused on 
a compact camera. Since this flat half mirror was 
mounted at 45°, the scale of the apparatus tended to 
become large. In addition, the seethrough optical sys- 
tem was unable to be established, because there was 
the eyepiece in front of the eye. 
[0005] Japanese Laid-open Patent Application No. 5- 
303053 discloses the structure that permits the 



seethrough optical system to be established, but the 
optical system was complex and the apparatus tended 
to become very large. 

[0006] Further, in cases where a reflective liquid crys- 
5 tal panel was used as a display element a separate illu- 
mination optical system had to be mounted and thus the 
scale of the overall apparatus tended to become large. 
[0007] With the image observing apparatus including 
the HMDs and the like, a significant challenge is to 
10 decrease the scale and weight of the overall apparatus, 
because the apparatus is mounted on the head of the 
observer. There are also desires for compact apparatus 
having no parallax and permitting establishment of the 
seethroutfi optical system. 

is 

SUMMARY OF THE INVENTION 

[0008] An object of the present invention is, therefore, 
to provide a composite display apparatus that permits 

20 the observer to observe the image information of the 
outside and the image information displayed on the dis- 
play element both in a good condition while being con- 
structed in compact size as a whole, by properly setting 
the structure of the image pickup optical system for 

25 imaging the image information of the outside on the sur- 
face of the image pickup device such as a CCD sensor 
and the structure of the display optical system including 
a prism body having a plurality of decentered reflective 
surfaces for guiding the light from the display element to 

so the eye of the observer on the occasion of observation 
of the image information displayed on the display ele- 
ment such as the liquid crystal display or the like. 
[0009] A composite display apparatus of the present 
invention is: 

35 

(1-1) a composite display apparatus comprising 
display means for displaying image information, a 
display optical system for guiding light from the dis- 
play means to the eye of an observer, and an image 

40 pickup optical system for focusing light from the out- 
side on an image pickup device, wherein an optical 
axis of the image pickup optical system is substan- 
tially aligned with an axis as an extension of the 
eye-ball optical axis in an optical path of the cfisplay 

45 optical system, the apparatus further comprises 
optical path separating means for separating an 
optical path of the image pickup optical system from 
the optical path of the cfisplay optical system, each 
of the display optical system and the image pickup 

so optical system has a plurality of decentered reflec- 
tive surfaces having curvature, the light from the 
cfisplay means is alternately folded in directions 
along the eye-ball optical axis by the plurality of 
reflective surfaces including the optical path sepa- 

55 rating means, of the display optical system to be 
guided to the eye, the light from the outside is alter- 
nately folded in directions different from an obser- 
vation direction by said plurality of reflective 
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surfaces including said optical path separating 
means, the image pickup optical system has a stop 
and a rear optical system placed behind the stop, 
the image pickup optical system comprises a sur- 
face having a negative power and a surface having 5 
a positive power in order toward the stop in the opti- 
cal path before said stop, and said rear optical sys- 
tem is a refracting optical system or a reflecting 
optical system having a positive power. 

Particularly, the composite display apparatus is 10 
characterized: 

(1-1-1) in that said image pickup optical system 
comprises a refractive surface having a nega- 
tive power and a reflective surface having a is 
positive power; 

(1-1-2) in that said image pickup optical system 
comprises a reflective surface having a nega- 
tive power and a reflective surface having a 
positive power; 20 
(1-1-3) in that said plurality of decentered 
reflective surfaces having curvature are sur- 
faces having powers differing depending upon 
azimuths about the vertex of surface; 
(1-1-4) in that said display optical system and 2s 
said image pickup optical system comprises a 
prism having three or more different optical sur- 
faces each having a power; 
(1-1-5) in that one surface of said image pickup 
optical system is a surface forming said optical 30 
path separating means; 
(1-1-6) in that said image pickup optical system 
is arranged to adjust focus by use of said rear 
optical system; 

(1-1-7) in that a reflective surface of said image 35 
pickup optical system, a reflective surface of 
said display optical system, and said optical 
path separating means are comprised of a 
common surface; 

(1-1-8) by comprising a seethrough optical sys- 40 
tern for guiding the light from the outside 
through said optical path separating means to 
the eye so as to permit the observer to observe 
the outside; 

(1-1-9) by comprising transmitting means for 45 
transmitting the image information from said 
image pickup optical system to said display 
means; 

(1-1-10) in that a view angle of an image of the 
outside displayed through said transmitting so 
means is equal to or smaller than a view angle 
at which the light from the outside is incident 
directly to the eye; 

(1 - 1 -1 1 ) in that said display means comprises a 
reflective display element the apparatus com- ss 
prises an illumination optical system for guiding 
illumination light to the display element the illu 1 
mi nation optical system shares a partial optical 
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system with said image pickup optical system 
and said display optical system, the illumina- 
tion optical system is separated from the opti- 
cal path of said display optical system by said 
first optical path separating means, and the illu- 
mination optical system is separated from said 
image pickup optical system by second optical 
path separating means; 
(1-1-12) in that at least one of said first and 
second optical path separating means is a 
polarization beam splitter; 
(1 -1 -13) in that said display optical system has 
a polarizing plate; 

(1-1-14) in that the polarizing plate of said dis- 
play optical system is rotatable; 
(1-1-15) in that said illumination optical system 
has a polarizing plate; 

(1-1-16) in that said image pickup optical sys- 
tem has a polarizing plate; 
(1-1-17) in that the polarizing plate of the image 
pickup optical system is rotatable; and so on: 

Another composite display apparatus of the 
present invention is: 

(1-2) a composite display apparatus comprising a 
(fisplay optical system having a plurality of reflective 
surfaces for guiding light from cfisplay means dis- 
playing image information to the eye of an observer, 
and an image pickup optical system having a plural- 
ity of reflective surfaces for focusing light from the 
outside on an image pickup device, wherein light 
passing through the image pickup optical system is 
separated in a reverse direction from an optical 
path of the display optical system by optical path 
separating means comprised of a decentered 
reflective surface having curvature, provided in the 
optical path, the display optical system comprises 
one reflective side of the reflective surface of the 
optical path separating means, the image pickup 
optical system comprises the other reflective side of 
the reflective surface of the optical path separating 
means, the image pickup optical system comprises 
a reflective surface having a positive power, a 
reflective surface having a negative power, a stop, 
and a rear optical system having a positive refrac- 
tive power, and focus is achieved by use of the rear 
optical system. 

Particularly, the composite display apparatus is 
characterized: 

(1 -2-1) in that said display means is illuminated 
with light from a light source by making use of 
said image pickup optical system and display 
optical system; 

(1 -2-2) in that the light from said light source is 
guided through a light splitting member to said 
image pickup optical system; and so on. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
[0010] 

Fig. 1 is a schematic diagram to show the main part 
of Embodiment 1 of the present invention; 
Fig. 2A and Fig. 2B are explanatory diagrams to 
illustrate modifications obtained by mocfifying part 
ofRg. 1; 

Fig. 3 is an explanatory diagram to illustrate a mod- 
ification obtained by modifying part of Fig. 1 ; 
Fig. 4 is an explanatory diagram to illustrate a mod- 
ification obtained by modifying part of Fig. 1 ; 
Fig. 5 is an explanatory diagram to illustrate a mod- 
ification obtained by modifying part of Fig. 1 ; 
Fig. 6 is an explanatory diagram to illustrate a mod- 
ification obtained by modifying part of Fig. 1; 
Fig. 7 is an explanatory diagram to illustrate a mod- 
ification obtained by modifying part of Fig. 1 ; 
Fig. 8 is a schematic diagram to show the main part 
of Embodiment 2 of the present invention; 
Fig. 9A and Fig. 9B are explanatory diagrams to 
illustrate modifications obtained by mocfifying part 
of Fig. 8; 

Fig. 10A and Fig. 10B are explanatory cGapams to 
illustrate modifications obtained by mocfifying part 
of Fig. 8; and 

Fig. 11 is a schematic diagram to show the main 
part of Embodiment 3 of the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0011] Fig. 1 is a schematic diagram to show the main 
part of Embodiment 1 of the present invention. The 
composite display apparatus of Fig. 1 has a display opti- 
cal system 10 composed of a display element (display 
means) 11 and a prism body L1, and an image pickup 
optical system 20 composed of a prism body L2, a stop 
23. a lens unit 24, and an image pickup device 25. 
[0012] Light emitted from the display element 1 1 such 
as a transmission type liquid crystal panel equipped 
with a back light is refracted by a surface 12 of the prism 
body L1 to be incident into the prism body L1 . Then the 
light is incident at an angle equal to or greater than the 
critical angle to a surface 13 to be totally reflected. The 
reflected light is further reflected by a mirror surface 14 
having a positive power to be again incident at an angle 
below the critical angle to the surface 13. The light is 
refracted to leave the prism body L1 and be guided to 
the observing eye E. The display optical system is 
arranged as described above. The three surfaces 12, 
13. and 14 of the prism body L1 have respective optical 
powers and form an enlarged virtual image of the dis- 
play element 1 1 in front of the observer. The surface 12, 
the surface 13, and the surface 14 are rotational ty 
asymmetric, aspherical surfaces having different pow- 
ers depending upon azimuths about the vertex of each 



surface, whereby even the small number of optical ele- 
ments can correct various aberrations caused by 
decentering of the optical system. If the surface 14 is a 
haff mirror surface, the display information of the display 
5 element 1 1 can be observed as spatially being superim- 
posed on the image information of the outside (an out- 
side image) through the prism body L2 described 
hereinafter. 

[Q01 3] The light from the outside is refracted by a sur- 

io face 21 of the prism body L2 to be incident into the 
prism body L2. Then the light is reflected by the mirror 
surface 14 having a negative power to be again incident 
at an angle over the critical angle to the surface 21 hav- 
ing a positive power to be totally reflected. The reflected 

is light is refracted by a surface 22 to leave the prism body 
L2 and be guided to the stop 23. The light passing 
through the stop 23 is collected by the lens unit 24 hav- 
ing a positive power to form the image information of the 
outside (the outside image) on the image pickup device 

20 25 such as the CCD sensor or the like. 

[0014] The elements 12, 23 to 25 descrfced above 
compose the image pickup optical system. The outside 
image obtained by the image pickup optical system is 
guided through an electric processing system (not illus- 

25 trated) to be displayed on the display element 1 1 as it is 
or after converted into some image information. Then 
the observer observes the display image. The surface 
21 and the surface 22 are rotationally asymmetric, 
aspherical surfaces having different powers depending 

30 upon azimuths about the vertex of each surface. This 
structure permits the positive power of the overall sys- 
tem to be dispersed with the stop 23 in between, 
thereby realizing the well-balanced image pickup optical 
system in terms of correction for aberration and the size 

35 of the optical system. 

[0015] The optical systems can be constructed in the 
compact size because of the above-stated structure in 
which the optical paths of the display optical system and 
the image pickup optical system are separated by using 

40 the front and back of the mirror surface 14 having the 
power (the surface 14 thus composing the optical path 
separating means for separating the optical paths) and 
in which the optical paths are folded in reverse direc- 
tions. The plurality of decentered reflective surfaces 

45 having curvature are made as prism surfaces in the ' 
integral form, which facilitates production. 
[001 6] The optical axis 01 of the display optical sys- 
tem is substantially aligned with the optical axis 02 of 
the image pickup optical system as illustrated in Fig. 1 

so and the view angle of the outside image picked up by 
the image pickup optical system display is equal to or 
smaller than the view angle at which the light from the 
outside directiy enters the eye, thereby enabling the dis- 
play and observation of a natural outside image without 

55 parallax. 

[0017] ff a synthesizing device not illustrated is used 
to synthesize the outside image picked up by the image 
pickup optical system and the display information gen- 
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erated by an information generating device not illus- 
trated to display a synthesized image on the display 
element 11, no parallax will be encountered and thus 
the superimposed information can be seen as rf rt is of a 
real world. 5 
[001 8] Fig. 2A is an explanatory diagram to illustrate a 
configuration in which the mirror surface 14 of Fig. 1 is 
replaced by a half mirror surface. When the mirror sur- 
face 14 is a half mirror as illustrated in Fig. 2A and when 
the refractive indices of the prism body L1 and prism 10 
body 12 are approximately equal, the display apparatus 
is provided with the seethrough function to permit the 
observer to observe the outside directly. Fig. 2B is an 
explanatory diagram to illustrate a configuration in 
which a lens LF is provided outside the prism body 12 of 75 
Fig. 1. The lens LF shown in Fig. 2B may have a power 
which is either positive or negative. 
[0019] Fig. 3 is an explanatory diagram to illustrate a 
configuration in which the lens unit 24 of the image 
pickup optical system of Fig. 1 is replaced by a reflective 20 
surface 24* having a positive power. 
[0020] In the present embodiment the focus of the 
image pickup optical system is adjusted by the lens unit 
after the stop surface 23. This is because focusing is 
easier with the smaller optical element 24, 24' after the 2s 
stop surface than the other optical elements. 
[0021 ] Fig. 4 to Fig. 7 are schematic diagrams to show 
the main part of modifications obtained by modifying 
part of Embodiment 1 of the present invention. In Fig. 4 
to Fig. 7 portions having the same functions as in the so 
embodiment illustrated in Fig. 1 are denoted by the 
same reference symbols and will be omitted from the 
description. 

[0022] The modification illustrated in Fig. 4 has the 
same structure as the embodiment illustrated in Fig. 1 as 
except that an optical element LR is added between the 
prism body L2 and the stop 23 in the image pickup opti- 
cal system 20 of the embodiment illustrated in Fig. 1. 
This structure permits the powers of the respective sur- 
faces forming the prism body L2 to be set smaller, 40 
thereby enabling to provide a more compact optical sys- 
tem. 

[0023] The modification illustrated in Fig. 5 has the 
■ same structure as the embodiment illustrated in Fig. 1 
except that a prism body LM is provided between the 45 
prism body L1 of the display optical system 10 and the 
prism body L2' of the image pickup optical system 20. 
as the optical path separating means of the embodi- 
ment shewn in Fig. 1 . When the reflective surface 14 of 
the display optical system 10 is separated from the so 
reflective surface 26 of the image pickup optical system 
20 as described, degrees of freedom of design are 
increased and thus the display optical system 10 and 
the image pickup optical system 20 can be corrected 
independently for aberration, thereby enabling to pro- ss 
vide a composite display apparatus with higher perform- 
ance. Here, the surface 14 of the prism body LM 
constitutes the optical path separating means of the dis- 



play optical system 1 0, while the surface 26 of the prism 
body LM does the optical path separating means of the 
image pickup optical system 20. 
[0024] The modification illustrated in Fig. 6 employs a 
prism body LW as the optical path separating means, 
as in the case of the embodiment shown in Fig. 5. This 
modification has the same structure as the modification 
illustrated in Fig. 5 except that the outside surface 21 ' of 
the prism body L2 v has a negative power and the sur- 
face 26' on the eye side has a positive power. The light 
from the outside is refracted by the surface 21' having 
the negative power to be incident to the prism body 12". 
Then the light is reflected by the mirror surface 26' hav- 
ing the positive power to be again incident at an angle 
over the critical angle to the surface 21' to be totally 
reflected thereby. The light is then refracted by the sur- 
face 22 to emerge from the prism body L2"and be 
guided to the stop 23. The optical path thereafter is the 
same as in the embodiment illustrated in Fig. 1. Here, 
the surface 14 of the prism body LM' constitutes the 
optical path separating means of the display optical sys- 
tem 1 0, while the surface 26' of the prism body LM' does 
the optical path separating means of the image pickup 
optical system 20. 

[0025] In the modification illustrated in Fig. 7, the dis- 
play optical system 1 0 and the image pickup optical sys- 
tem 20 are constructed of reflective mirror units and 
lens units. The light emitted from the display element 1 1 
is refracted by a lens unit 17, is reflected by a half mirror 
15 and an eye-side surface 16a of an mirror 16, and 
passes through the half mirror 15 to be guided to the 
eye E. The light from the outside passes through a half 
mirror 27, is reflected by an outside surface 16b of the 
mirror 16 having a negative power, and is reflected by 
the half mirror 27 having a negative power to be guided 
to the stop 23. The optical path thereafter is the same as 
in the embodiment illustrated in Fig. 1 . Here, the surface 
1 6a of the mirror 1 6 constitutes the optical path separat- 
ing means of the cGsplay optical system 10, while the 
surface 16b of the mirror 16 does the optical path sepa- 
rating means of the image pickup optical system 20. 
[0026] In the modifications illustrated in Fig. 5 and Fig. 
6, the prism bodies LM and LM' may be made of an 
adhesive. 

[0027] Fig. 8 is a schematic dagram to show the main 
part of Embodiment 2 of the present invention. The 
composite display apparatus illustrated in Fig. 8 has the 
display optical system 30 composed of a reflective dis- 
play element 31 and a prism body L3, the image pickup 
optical system 40 composed of a prism body L4, a 
polarization beam splitter 43, a stop 44, a lens unit 45, 
and an image pickup device 46, and an illumination opti- 
cal system 50 composed of the prism body L3 and 
prism body L4, the polarization beam splitter 43, and a 
light source 51. 

[0028] An optical thin fOm with polarization selectivity 
as first optical path separating means is formed at the 
interface 34 between the prism body L3 and the prism 
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body L4. The surface 34 and the polarization beam 
splitter 43 as second optical path separating means 
have such a property as to transmit the component of 
linearly polarized light whose direction of vibration of the 
electric vector is parallel to the plane of the drawing 5 
(hereinafter referred to as p-polarized light) but to reflect 
the component of linearly polarized light whose direc- 
tion of vbration of the electric vector is normal to the 
plane of the drawing (hereinafter referred to as s-pdar- 
ized light). The prism body L3 and prism body L4 are w 
made of an optical material having little or no birefrin- 
gence. 

[0029] The light emitted from the light source 51 such 
as an LED passes through the polarization beam split- 
ter 43 to become a beam of only p-polarized light The is 
p-polarized light is refracted by a surface 42 to enter the 
prism body L4. The incident light is incident at an angle 
over the critical angle to the surface 41 to be totally 
reflected. The reflected light passes through the surface 
34 to enter the prism body L3. Then the light is incident 20 
at an angle over the critical angle to the surface 33 to be 
totally reflected. The reflected Bght is emergent from the 
prism body L3 as being refracted by the surface 32, 
then illuminating the display element 31 . 
[0030] The display element 31 is a reflective display 2s 
element, for example, like a reflective liquid crystal 
panel, which has such a reflection property as to rotate 
the vibration direction of the electric vector of the inci- 
dent light 90° where the cfisplay is "white" but to retain 
the vbration cfirection where the display is "black". 30 
[0031] The light reflected by the display element 31 
under the illumination by the illumination optical system 
50 is incident to the prism body L3 as being refracted by 
the surface 32 of the prism body L3. Then the light is 
incident at an angle over the critical angle to the surface ss 
33 to be totally reflected. The reflected light is guided to 
the interface 34 having a positive power. Since the 
plane of polarization of reflected light from pixels in the 
"write" state in the display element 31 was rotated 90°, 
the reflected light became s-polarized light. Therefore, it 40 
is reflected by the interface 34 to be again incident at an 
angle below the critical angle to the surface 33 to be 
refracted by and be emergent from the prism body 13. 
The light is thus guided to the observing eye E. 
[0032] On the other hand, the reflected light from pix- 4s 
els in the "Mack" state in the display element 31 is 
retained as p-polarized light Therefore, it is transmitted 
by the interface 34, is reflected by the surface 41, is 
refracted by the surface 42 to leave the prism body L4, 
and is transmitted by the polarization beam splitter 43. so 
Therefore, the reflected light from the display element 
31 is not incident to the image pickup device 46, so as 
to be prevented from being projected thereonta 
[0033] The s-polarized light component among the 
light from the outside is refracted by the surface 41 hav- ss 
ing a positive power to enter the prism body L4, is 
reflected by the interface 34 having a negative power, is 
again incident at an angle over the critical angle to the 



surface 41 to be totally reflected, is refracted by the sur- 
face 42 to leave the prism body L4, and is reflected by 
the polarization beam splitter 43 to be guided to the stop 
44. The light passing through the stop 44 is collected by 
the lens unit 45 having a positive power to form the out- 
side image on the image pickup device 46 such as the 
CCD sensor. 

[0034] On the other hand, the p-polarized light compo- 
nent is incident from the surface 41 into the prism body 
L4, is transmitted by the interface 34 to enter the prism 
body L3, and is emitted from the surface 33 to be guided 
to the observing eye E. The seethrough optical system 
is thus established. In this case the seethrough optical 
system is arranged to have substantially no power by 
optimizing the shapes of the surface 41 and surface 33. 
[0035] Fig. 9A and Fig. 9B are examples in which a 
polarizing plate PL is placed on the outside of the prism 
body L4 or on the inside of the prism body L3 so as to 
transmit only the s-polarized fight. This can intercept the 
light so as to prevent the light from the outside from 
reaching the eye E. Further, a rate of quantity of light 
from the display element and quantity of light from the 
outside can be adjusted by controlling an angle of the 
polarizing plata 

[0036] In this structure the polarization beam splitter 
43 is placed in front of the stop 44 of the image pickup 
optical system, whereas the polarization beam splitter 
43 may also be placed after the rear optical system 45 
of the image pickup optical system 40, as illustrated in 
Rg. 10A and Fig. 10B. Fig. 10A shows an example in 
which a surface illuminant 47 is used as a light source, 
while Rg. 10B shows an example in which an almost 
point source 48 such as the LED is used in combination 
with a condenser lens 49. 

[0037] Rg. 11 is a schematic diagram to show the 
main part of Embodiment 3 of the present invention. In 
Rg. 11 elements having the same functions as those in 
Embodiment 2 of Rg. 8 are denoted by the same 
numerals. The present embodiment is different from 
Embodiment 2 in that a polarizing plate 35 is provided in 
the display optical system 30, the interface between the 
prism bodies L3 and L4 is a half mirror (first optical path 
separating means) 36, and the second optical path sep- 
arating means for separating the optical paths of the illu- 
mination optical system 50 and the image pickup optical 
system 40 is constructed of a half mirror 47. 
[0038] The light emitted from the light source 51 such 
as the LED is transmitted by the half mirror 47 and is 
refracted by the surface 42 of the prism body L4 to enter 
the prism body L4. The incident light is incident at an 
angle over the critical angle to the surface 41 to be 
totally reflected, is transmitted by the surface 36 to enter 
the prism body L3. and is incident at an angle over the 
critical angle to the surface 33 to be totally reflected. 
The reflected light is emergent from the prism body L3 
as being refracted by the surface 32. The light then 
passes through the polarizing plate 35 to become, for 
example, only the p-polarized light component, thus illu- 
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minating the display element 31 . 
[0039] The display element 31 is a reflective display 
element, for example, like a reflective liquid crystal 
panel, which has such a reflection property as to rotate 
the vibration direction of the electric vector of the inct- s 
dent light 90° where the display is "black" but to retain 
the vbration cfirection where the display is "white". 
[00401 The light reflected by the display element 31 
under the illumination by the illumination optical system 
50 is incident to the polarizing plate 35 and only light w 
from "white" pixels in the display, the direction of polari- 
zation of which was retained, is transmitted thereby. The 
light from the polarizing plate 35 is incident to the prism 
body L3 as being refracted by the surface 32. Then the 
light is incident at an angle over the critical angle to the is 
surface 33 to be totally reflected. The reflected light is 
guided to the half mirror 36 having a positive power to 
be reflected. The reflected light is again incident at an 
angle below the critical angle to the surface 33 to be 
refracted and emitted from the prism body L3, and it is 20 
then guided to the observing eye E. 
[0041] The light from the outside is refracted by the 
surface 41 having a positive power to enter the prism 
body L4. The incident light is reflected by the half mirror 
36 having a negative power and is again incident at an 25 
angle over the critical angle to the surface 41 to be 
totally reflected. The reflected fight is refracted by the 
surface 42 to leave the prism body L4. The light is then 
reflected by the half mirror 47 to be guided to the stop 
44. The light passing through the stop 44 is collected by 30 
the lens unit 45 having a positive power to form the out- 
side image on the image pickup device 46 such as the 
CCD sensor. 

[0042] On the other hand, a beam of the part transmit- 
ted by the half mirror 36 is incident to the prism body L3, 35 
is emergent from the surface 33, and is guided to the 
observing eye E. This thus forms the seethrough optical 
system. In this case the seethrough optical system is 
arranged to have substantially no power by optimizing 
the shapes of the surface 41 and the surface 33. 40 
[0043] Among the light from the display element 31 , 
the beam of the part transmitted by the half mirror 36 
reaches the image pickup device 46, but the display 
image can be prevented from being picked up, by driv- 
ing the display and image pickup operations in time divi- as 
sion or by interposing a polarizing plate in the image 
pickup optical system in the relation of crossed Nicols 
with the polarizing plate 35. 

[0044] In Embodiment 3 the optical path separating 
means may also be placed after the rear optical system so 
45 of the image pickup optical system 40, as stated in 
Embodiment 2 (Fig. 10A and Fig. 10B). Further, 
Embodiments 2 and 3 can also employ the configura- 
tion using half mirrors for the surface 33 and surface 41 
as in Embodiment 1 , the configuration using a reflective ss 
surface for, the rear optical system of the image pickup 
optical system (Fig. 3), the configuration using a lens 
placed before the prism body (Fig. 2B), or the configura- 



tions as illustrated in Fig. 4 to Fig. 7. 
[0045] The present invention can provide the compact 
composite display apparatus according to the above 
structure in which the seethrough optical system can 
also be established in the type without parallax and the 
compact composite display apparatus even if the reflec- 
tive liquid crystal panel is used as a display element 
[0046] In addition, the present invention can also 
achieve the composite display apparatus permitting the 
image information of the display element and the image 
information of the outside both in a good concfition while 
achieving compactif ication of the overall apparatus, by 
properly adjusting the structure of the image pickup 
optical system for imaging the image information from 
the outside on the surface of the image pickup device 
such as the CCD sensor and the structure of the (fisplay 
optical system including the prism body having the plu- 
rality of decentered reflective surfaces for guiding the 
light from the display element to the observer's eye on 
the occasion of observation of the image information 
displayed on the display element such as the liquid crys- 
tal display. 

[0047] A composite display apparatus has a display 
device for displaying image information, a display opti- 
cal system for guiding light from the display device to 
the eye of an observer, and an image pickup optical sys- 
tem for focusing light from the outside on an image 
pickup device. An optical axis of the image pickup opti- 
cal system is substantially aligned with an axis as an 
extension of the eye-bail optical axis in an optical path of 
the display optical system. The apparatus further has an 
optical path separating element for separating an optical 
path of the image pickup optical system from the optical 
path of the display optical system. Each of the (fisplay 
optical system and the image pickup optical system has 
a plurality of decentered reflective surfaces having cur- 
vature. The light from the display device is alternately 
folded in directions along the eye-ball optical axis by the 
plurality of reflective surfaces of the display optical sys- 
tem to be guided to the eye. The light from the outside is 
alternately folded in directions different from an obser- 
vation direction by the plurality of reflective surfaces 
including the optical path separating element The 
image pickup optical system has a stop and a rear opti- 
cal system placed behind the stop, and the image 
pickup optical system has a surface having a negative 
power and a surface having a positive power in order 
toward the stop in the optical path before the stop The 
rear optical system is a refracting optical system or a 
reflecting optical system having a positive power. 

Claims 

1 . A composite display apparatus comprising (fisplay 
means for displaying image information, a (fisplay 
optical system for guiding light from the (fisplay 
means to the eye of an observer, and an image 
pickup optical system for focusing light from the out- 
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side on an image pickup device, wherein an optical 
axis of the image pickup optical system is substan- 
tially aligned with an axis as an extension of the 
eye-ball optical axis in an optical path of the display 
optical system, the apparatus further comprises 5 
optical path separating means for separating an 
optical path of the image pickup optical system from 
the optical path of the display optical system, each 
of the display optical system and the image pickup 
optical system has a plurality of decentered reflec- w 
tive surfaces having curvature, the light from the 
display means is alternately folded in directions 
along the eye-ball optical axis by the plurality of 
reflective surfaces including the optical path sepa- 
rating means, of the display optical system to be 75 
guided to the eye, the light from the outside is alter- 
nately folded in (fractions different from an obser- 
vation direction by said plurality of reflective 
surfaces including said optical path separating 
means, the image pickup optical system has a stop 20 
and a rear optical system placed behind the stop, 
the image pickup optical system comprises a sur- 
face having a negative power and a surface having 
a positive power in order toward the stop in the opti- 
cal path before said stop, and said rear optical sys- 25 
tern is a refracting optical system or a reflecting 
optical system having a positive power. 

2. The composite display apparatus according to 
Claim 1, wherein said image pickup optical system 30 
comprises a refractive surface having a negative 
power and a reflective surface having a positive 
power. 

3. The composite display apparatus according to as 
Claim 1 , wherein said image pickup optical system 
comprises a reflective surface having a negative 
power and a reflective surface having a positive 
power. 

40 

4. The composite display apparatus according to 
Claim 1, 2, or 3, wherein said plurality of decen- 
tered reflective surfaces having curvature are sur- 
faces having powers differing depending upon 
azimuths about the vertex of surface. 45 

5. The composite display apparatus according to 
Claim 1 , 2, 3, or 4, wherein said display optical sys- 
tem and said image pickup optical system com- 
prises a prism having three or more different optical so 
surfaces each having a power. 

6. The composite display apparatus according to 
either one of Claims 1 to 5, wherein one surface of 
said image pickup optical system is a surface form- ss 
ing said optical path separating means. 

7. The composite display apparatus according to 



either one of Claims 1 to 6, wherein said image 
pickup optical system is arranged to adjust focus by 
use of said rear optical system. 

8. The composite display apparatus according to 
either one of Claims 1 to 7, wherein a reflective sur- 
face of said image pickup optical system, a reflec- 
tive surface of said display optical system, and said 
optical path separating means are comprised of a 
common surface. 

9. The composite display apparatus according to 
either one of Claims 1 to 8, comprising a 
seethrough optical system for guiding the light from 
the outside through said optical path separating 
means to the eye so as to permit the observer to 
observe the outside. 

10. The composite display apparatus according to 
erther one of Claims 1 to 9, comprising transmitting 
means for transmitting the image information from 
said image pickup optical system to said display 
means. 

11. The composite display apparatus according to 
Claim 10, wherein a view angle of an image of the 
outside displayed through said transmitting means 
is equal to or smaller than a view angle at which the 
light from the outside is incident directly to the eye. 

12. The composite display apparatus according to 
either one of Claims 1 to 11, wherein said display 
means comprises a reflective display element the 
apparatus comprises an illumination optical system 
for guiding illumination light to the display element, 
the illumination optical system shares a partial opti- 
cal system with said image pickup optical system 
and said display optical system, the illumination 
optical system is separated from the optical path of 
said display optical system by said first optical path 
separating means, and the illumination optical sys- 
tem is separated from said image pickup optical 
system by second optical path separating means. 

13. The composite display apparatus according to 
Claim 12, wherein at least one of said first and sec- 
ond optical path separating means is a polarization 
beamsplitter. 

14. The composite display apparatus according to 
Claim 12 or 13, wherein said cfisplay optical system 
has a polarizing plate. 

15. The composite display apparatus according to 
Claim 14, wherein the polarizing plate of said cfis- 
play optical system is rotatabJe. 

16. The composite display apparatus according to 
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either one of Claims 1 2 to 1 5, wherein said illumina- 
tion optical system has a polarizing plate. 

17. The composite display apparatus according to 
either one of Claims 12 to 16. wherein said image s 
pickup optical system has a polarizing plate. 

18. The composite display apparatus according to 
Claim 17, wherein the polarizing plate of the image 
pickup optical system is rotatable. io 

19. A composite cfisplay apparatus comprising a dis- 
play optical system having a plurality of reflective 
surfaces for guiding fight from display means dis- 
playing image information to the eye of an observer, is 
and an image pickup optical system having a plural- 
ity of reflective surfaces for focusing light from the 
outside on an image pickup device, wherein light 
passing through the image pickup optical system is 
separated in a reverse direction from an optical 20 
path of the display optical system by optical path 
separating means comprised of a decentered 
reflective surface having curvature, provided in the 
optical path, the display optical system comprises 
one reflective side of the reflective surface of the 2s 
optical path separating means, the image pickup 
optical system comprises the other reflective side of 
the reflective surface of the optical path separating 
means, the image pickup optical system comprises 

a reflective surface having a positive power, a 30 
reflective surface having a negative power, a stop, 
and a rear optical system having a positive refrac- 
tive power, and focus is achieved by use of the rear 
optical system. 

35 

20. The composite display apparatus according to 
Claim 19, wherein said display means is illuminated 
with light from a light source by making use of said 
image pickup optical system and display optical 
system. ao 

21. The composite display apparatus according to 
Claim 20, wherein the light from said light source is 
guided through a light splitting member to said 
image pickup optical system. as 
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FIG. 2A 




FIG. 2B 
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FIG. 9 A 




FIG. 9B 




EP0982 614 A2 



FIG. 10A 




16 



EP0 982 614 A2 



FIG. 11 
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